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ABSTRACT 


Teleoperation of land vehicles secu- 

capability U These vehiclerhave chai^te^ 

visual systems. Experimentation studyi g versus steering-coupled camera 

INTRODUCTION 


the remainder of this paper) is gene y PF operation is increased com- 

SSK rrio»“-o« comen, of Ure o,*ra,or feedback is degraded by the 
intermediary channels. 


lit,,, work performed at Sandia National Laboratories support by the U.S. Depanmen, of Energy under 
contract number DE-AC04-76DP00789. 
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Vehicles, control stations, and teleoperated systems have been built, tested, and demon- 
strated by a number of organizations. There is little definitive information, however, on the human 
factors involved m land vehicle teleoperation (ref. 1). Most information has taken the form of a 
description of vehicle design or proposed application, with only a few papers reporting actual 
experimental results. Most of the knowledge base is represented by personal experiences and 
unreported anecdotal evidence. This paper attempts to expand the data base through a presentation 
of some of the preliminary results of experimentation in teleoperation at Sandia National Laborato- 
ries and through discussion of the observations of Sandia personnel gathered over several years of 
teleoperation experience. 


TELEOPERATION SYSTEMS 


Sandia National Laboratories has been actively studying teleoperation for several years 
The major effort has entailed the development of a fleet of wheeled vehicles ranging in size from 
small, interior test beds to large, road and off-road commercial and military vehicles (ref. 2). 
These vehicles (shown m fig. 1) are being used to conduct feasibility studies on the application of 
teleoperated vehicles to the physical security and military needs of the U.S. Government. In all of 
mese vehicles, actuators operate the vehicle throttle, brakes, and steering. Control may be derived 
trorn manual input at a remote driving station or through some level of automatic control from a 
digital computer. On-board processing may include simple vehicle control functions or may allow 
for unmanned, autonomous operation. Communication links are provided for digital communica- 
tion between control computers, television transmission for vehicle vision, and voice for local 
control* 


o Comroi stanons have been developed to support remote operation of the Sandia vehicle 
fleet. Capabilities range from single television monitor stations with vehicle feedback limited to an 
audio channel (shown in fig. 2), through large, multiscreen, panoramic displays with computer- 
generated graphics representations of vehicle speed, pitch, roll, and heading (fig. 3). Vehicle 
camera mountings have included a single fixed camera, multiple fixed cameras, and cameras slaved 
to the vehicle steering gear. To date, Sandia has not experimented with stereo vision or with head- 
slaved displays, although members of the staff have operated such equipment at other locations. 

u u ., Y nder * e s P° nsorshi P of the U.S. Army Missile Command, through the Teleoperated 
Mobile Antiarmor Platform (TMAP) Project, Sandia has embarked on a major set of experiments to 
verify some of the observations regarding the "best" driving display (ref. 3). In particular the 
experimentation addresses the problems of detection and identification of obstacles in the path of 
the vehicle. Specific questions include the effect of color versus black and white, the utility of 
m creasing the horizontal field of view through panning a camera in response to steering wheel 
movements (steering-slaved control), and the errors in operator interpretation of size and distance 
information as presented by the television system. 


EXPERIENCE 

The experimentation on obstacle detection and vehicle control being performed for the 
TMAP Project represents the only rigorous data base development in process at Sandia. In this 
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STSS of di data analysis for .his series of tests has been completed (refs. 4 and 5). 
Additional tests and experimentation are being planned. 

The remainder of the experience base at Sandia has been derived from operation of vehicles 
ators and observers. 

The analysis of accidents involving teleoperated vehicles has provided additional informa- 
tion Table 1 nrovides a listing Some of these accidents occurred while the operator was observ- 
tan JveWclSdy (outside-in operation) and were predominately depdt-percepdon problems 

parked car. 


TABLE 1.- ACCIDENT HISTORY 


[ABLE i 

VFHin E 1 INCIDENT | CAUSE 

Outside-In Operation 

Dune Buggy 
Dune Buggy 
Dune Buggy 
Suzuki 
Suzuki 

Hit fence 
Hit tree 
Hit fence 
Hit post 
Hit car 

Underestimated stopping distance 
Depth perception 
Underestimated stopping distance 
Depth perception 
Control reversal 


Tnside-Out Operation 

Suzuki 

Suzuki 

Suzuki 

Suzuki 

Suzuki 

Suzuki 

Suzuki 

Suzuki 

Suzuki 

Rollover 

Rollover 

Rollover 

Rollover 

Rollover 

Rollover 

Rollover 

Rollover 

Rollover 

Loss of control on hill 
Loss of control on hill 
Hit traffic cone 

Loss of control on hill I 

Loss of control on hill 

Loss of control while backing 

Loss of control, hit bump 

Loss of control on hill 

Loss of control, hit bump 


All of the accidents involving teleoperation (inside-out control) have been rollovers. The 
particular^ehicle involved iTa str^lf Suzuki LT50 four-wheel, all-terrain veWcles^jmflg- 
nre 4 The rear wheels are driven through a single-speed drive with a centrifugal clutch. The 
v^icle^s candle <rfa 15^np^top speed on flat ground. Control inputs from the operator are 

though the control station illustrated in figure 2. Figure 5 shows the ^ 

In all but one incident, the vehicle was being operated off-road on a motor-cross track with 


38-3 



steep slopes, high banked comers, and high berms at the edges of the track. The only exception 
was a rollover caused by hitting a traffic cone while operating on a flat asphalt parking lot. 


OBSERVATIONS 


A number of observations regarding important parameters, operational considerations, and 
system design features have been derived from Sandia experiences. These are presented below 
strictly as indicators since, in the absence of hard experimental data, it is not clear that all are gen- 
erally applicable. Likewise, not all system implementations are represented. 


Field of View 

It is very difficult to operate a vehicle in restricted space with a narrow field of view. 
Operations of a Jeep Cherokee on normal roads and parking lots were performed with a single 
camera, 40° field-of-view system. The operator was not comfortable turning comers. Installation 
of two additional cameras, to provide a total of 120° field of view resulted in much "easier" opera- 
tion. Additional tests have been run using a steering-slaved camera, both on the Jeep Cherokee 
and on the Suzuki all-terrain vehicle. Steering-slaved viewing provided sufficient effective field of 
view to allow turning tight comers and avoiding obstacles. Provision of a mechanism to allow the 
operator to force the camera further (an auxiliary pan control) was even more effective. 


Resolution 


Camera resolution does not seem to be a factor in the ability to teleoperate a vehicle in the 
absence of obstacles. Sandia has operated vehicles with malfunctioning communications links 
resulting in extremely poor resolution. As long as operations take place on well defined areas 
(such as well marked roads) and there are no obstacles in the path of travel, an operator can suc- 
cessfully maneuver a vehicle from one point to another. High resolution does appear to be impor- 
tant when many sizes and types of obstacles are present and for operation off-road where identifi- 
cation of best path is important. 


Color/Black and White 


, . . Work with television surveillance systems has indicated that the increased resolution possi- 

ble with black and white equipment is much more important than any additional information con- 
ta “?. . in , c °!° r signal- This does not necessarily appear true for teleoperation. Color provides 

a ““ tt ° nal cues leadin g to more accurate obstacle recognition and course planning. For example, 
the difference between dirt and asphalt is important for driving, but cannot be determined from a 
black and white television picture. Sandia has also found that orange traffic cones (with the color 
chosen for maximum visibility) tend to disappear on black and white television. These have been 
used to establish courses during demonstrations and experimentation. Using black and white tele- 
vision, it was found to be necessary to cover the cones with white paper to so that they could be 
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Vehicle Vibration 


Vehicle vibration and bounce has no. been observed to 
video scene The small Suzuki has no suspension (spnngs or damping) other than its Inge, so 
off road tires During operations which lead to the vehicle bouncing enough to actually leave th 
"t^vidSa^ relatively clear and usable. No operator has ever commented that vibra- 
tion or bounce in the picture was bothersome. 


Distance Estimation 

As seen from the accident reports, distance estimation during £ T£ b a 

1 Tt «i Qn rr eates difficulties when using inside-out control. As reported by Spain (re . ) 

using a head-mounted display consistently ran mtopy ons 
u j Q norVina niare The feeline of being further from obstacles and landmarks 

better results, scene magnification of approximately 25% is required. 


Negative Obstacles 

Terrain features such as ditches, holes, and drop-offs are extremely difficult to see using 
television Negative obstacles such as these have contributed to many of the problems in teleoper- 
rOTl, small ditches cannot be differentiated from variations m ground col- 
oration until the vehicle has hit them. At that point, the horizon on the video ^ene ^ges^uidi- 
cating that the vehicle just hit a ditch. It can be anticipated that stereo vision could help in 
problem, but no experimentation has been reported. 


Tilt and Roll 

The large number of rollovers reported establish vehicle tilt and roll control as a major 
problem In th! Suzuki driving system, the only feedback is the video signal from the cameraand 
In audio pickup providing engine sound. Vehicle attitude parameters are neither measured nor 
3 Sal Occident scenario entails "launching" the vehicle from a ramp or attempting 

m ^averse a sid^lope which is too steep for the vehicle to maintain stability. Most rollovers have 

occurred at close to maximum vehicle speed (about 10-15 mph) i and ha ^ e ^.^ es ^^ 45 o 
features representative of extremely challenging terrain. These have included hills with up to 45 
slopes andTiighly banked corners on a motor-cross course. As the rollover occurs, the °P e ^ at ° 
express surprise * In debriefing, it appears that the operator had no indication that the vehicle was 

approaching a dangerous condition. 
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Overcontrol 


A typical characteristic of novice operators is extreme steering overcontrol. The operator 
applies a small steering input to the vehicle, but no result is immediately seen. The steering input is 
increased until a response is finally observed. The resulting turn is more than intended so the 
operator applies a small correction. Again, the response is not seen so more correction is applied, 
etc. The outcome is vehicle travel oscillating about the desired path. Operators report this to be a 
very stressful situation. Overcontrol has also contributed to several of the vehicle rollover acci- 
dents. The operator applied excessive steering input, sending the vehicle over the edge of a berm. 
Observing novice drivers learning to control the vehicle, it is apparent that considerable internal 
control is being exercised as the operator adapts. After some minutes of operation, steering opera- 
tion is considerably slower and at lower amplitude, resulting in smoother vehicle control. Spain 
(ref. 6) reports similar findings. 


Navigation 

An associated problem in vehicle teleoperation is the difficulty of maintaining spatial orien- 
tation with respect to major landmarks, map features, or compass directions. It is not uncommon 
for operators to become lost on the motor-cross course. Even with landmarks and a map of the 
course, they have not been able to determine how to return to the starting location without 
assistance. 


SUMMARY AND CONCLUSIONS 


Operational experience has been gathered at Sandia through development, test, and demon- 
stration of a number of vehicles. A large experimental program in vision system requirements for 
teleoperation is also in process. Through the knowledge gained in these programs, several key 
areas can be identified as critical to successful control of a teleoperated vehicle. The primary area is 
the quality of the visual display provided to the operator. It has been shown that vehicles can be 
controlled in restricted environments with extremely poor conditions of viewing. As viewing 
improves (both in resolution and field of view), better control can be expected. 

Negative obstacles create difficulty in that operators cannot distinguish them from other 
terrain features which do not affect vehicle travel. The result is hitting ditches, holes, or berms at 
excessive speed. 

The interaction of the vehicle with the environment, as interpreted through the mediating 
effects of the television display system, can lead to poor control capabilities and hazardous operat- 
ing conditions. Overcontrol of the vehicle steering, coupled with the operator's inability to accu- 
rately perceive vehicle attitude and terrain requirements has led to a number of accidents. This can 
be partially linked with the absence of kinesthetic feedback to the operator. Experimentation with 
vehicle simulators has shown a distinct lag in response to environmental inputs, such as wind 
gusts, when no kinesthetic feedback is present (ref. 8). With the addition of kinesthetic feedback 
to the operator (simulator platform motion), response time to sudden wind gusts dropped from an 
average of 0.56 sec to an average of 0.44 sec. Similar results have been reported for the addition 
of steering wheel torque feedback, thus providing "feel of the road" to the operator (ref. 9). The 
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lack of kinesthetic feedback is similar to operating with a time delay in the control system. 
Additional lags are introduced by the communications systems and vehicle actuator and control 
systems. 


Given the ability to maneuver a teleoperated vehicle in the real-world environment, the 
problem of navigation is encountered. Operators tend to get lost, disoriented, and confused when 
provided with visual input and maps. The effect of addition of vehicle heading, plotting of route 
traveled, or other aids remains to be investigated. 
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Figure 1 Teleoperated vehicles. 
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Figure 2.- Single monitor with audio feedback. 
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Figure 3- Panoramic display. 
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Figure 4.- All-terrain vehicle. ORIGINAL PAGE IS 
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Figure 5 - Operator's view via control station. 
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